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ABSTRACT 



This paper reviewed the research on critical thinking among 
college students. A total of 62 studies were identified and reviewed. About 
62 percent of the studies were longitudinal in nature, and 13 of the 23 
studies that attempted to measure student growth in critical thinking 
employed a cross-sectional design. While a large body of the findings 
suggested that students grow in critical thinking while in college, much 
inconsistency emerged as to the factors that affected this growth. Given that 
studies in this area tend to be homogenous to the extent that they pose the 
same research questions and employ similar research tools, a surprising 
amount of inconsistency emerged from the study results. A number of studies 
suggested that college students make the greatest gains in critical thinking 
during their freshman year, while courses or programs specifically designed 
to improve critical thinking have demonstrated mixed results. In comparison 
to courses taught in a more traditional manner, greater gains in critical 
thinking scores were found for courses with an instructional paradigm 
emphasizing problem solving or critical thinking, class participation, and 
inquiry and higher-order thinking. (Contains 83 references.) (MDM) 
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This review of research on critical thinking involves 62 studies. An overview of findings 
is presented, followed by an analysis of study methodology. While a large body of 
findings suggests that students grow in critical thinking while in college, much 
inconsistency has emerged as to the factors that affect this growth. Implications drawn 
for future research include the need to diversify research designs, measurement 
instruments, and variables investigated. 
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A Review of Research on Critical Thinking 

Cultivating critical thinkers is a major goal of American education. Pledges by 
higher education institutions to instill critical thinking skills in students is commonplace, 
as most college catalogs will substantiate. A number of well known national reports such 
as the National Education Goals Report (\99\), Integrity in the College Curriculum 
(1985), Involvement in Learning (1984), Academic Preparation for College (1983), and 
Action for Excellence (1983), extol the importance of enhancing students’ abilities to 
think critically and champion heightening efforts to meet this objective. Despite the 
widely professed centrality of critical thinking development to the educational enterprise, 
much uncertainty exists as to how educators can most effectively foster this valuable 
skill. 

McMillan’s 1987 review of 27 studies on critical thinking at the higher education 
level is considered by many to be the most significant overview of research on this 
subject. More than a decade has elapsed since that publication, and the pool of research 
studies on critical thinking has expanded significantly. A re-examination of the state of 
educational research on this important topic is thus again warranted. The present study 
analyzes 62 studies that investigate critical thinking among college students. Study 
results are organized in two sections. First, an overview of research findings generated 
by this rich sample of studies is presented. Second, analysis of methodological 
approaches leads to the address of some salient trends and considerations. 

The research studies included in this analysis, which are published in the form of 
journal articles, dissertations, and books, were identified through searches using the key 
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words “critical thinking” and “higher education” on such databases as Education 
Abstracts, ERIC (Educational Resources Information Center), and Dissertation Abstracts. 
While this sample of 62 studies is extensive, it is not exhaustive. Nonetheless, this 
sample constitutes a fair representation of the kinds of empirical investigations that have 
been conducted on critical thinking within the higher education setting. 

Critical Thinking Defined 

Because critical thinking is a complex skill, any attempt to offer a complete and 
definitive definition of it would be futile. Like other complex constructs, critical thinking 
seems to be easier for one to recognize than to define. And, while there is no universally 
accepted singular definition of critical thinking, there appears to be some agreement as to 
what the term generally refers to. Educational researchers who tackle the definitional 
question for the most part do not offer dramatically different interpretations of what 
critical thinking represents. As noted by Halpren (1993), although absolute agreement on 
what constitutes critical thinking does not exist, “there is sufficient overlap in the various 
definitions to allow an evaluator to move beyond the definitional stage.” 

In a review of the manner in which researchers operationalize critical thinking, 
Furedy and Furedy (1985) found that critical thinking is typically thought to involve an 
individual’s ability to do some or all of the following: identify central issues and 
assumptions in an argument; recognize important relationships; make correct inferences 
from the data; deduce conclusions from information or data provided; interpret whether 
conclusions are warranted based on given data; and, evaluate evidence or authority. 
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There have been several notable debates over the construct of critical thinking. 
One controversy pertains to whether critical thinking is a general skill or a subject- 
specific skill. According to McPeck (1985), critical thinking is not a content-free general 
ability even though most tests designed to measure critical thinking treat it as if it were. 
McPeck argues that because critical thinking is necessarily about some particular thing or 
subject, it must be a context-specific intellectual skill. As such this skill is dually based, 
consisting of both a knowledge component and a critical component. Those holding this 
point of view tend to advocate the teaching of critical thinking only as instruction within 
subject areas or disciplines. Others argue that while a student’s familiarity with subject 
matter undoubtedly affects a student’s performance on thinking tasks in that area, this 
does not rule out the existence of general principles of critical thinking that have wide 
applicability and transcend specific subjects (Ennis, 1987; Facione, 1990). 

King, Wood, and Mines (1990) point out that confusion over critical thinking also 
arises due to a failure to address an important distinction between “well-structured 
problems” and “ill-structured problems.” Problem structure is concerned with the 
certainty by which a solution to a problem can be judged true or correct. On a scale of 
absolute certitude, well-structured problems have solutions at the high end while ill- 
structured problems have solutions at the low end. Because the issue of problem 
structure is not directly addressed in many studies, the precision by which study 
implications are drawn is often compromised. Commonly utilized measurement 
instruments such as the Watson-Glaser Critical Thinking Appraisal (WGCTA), Cornell 
Critical Thinking Test (CCTT), and Reflective Judgment Interview (RJI), because they 
differ drastically in their inclusion of well- or ill- structured problems (King et. al, 1990), 




6 



4 



embody different criteria for critical thinking ability and can potentially yield divergent 
results. King et. al (1990) estimated the correlation between WGCTA and RJI to be .46, 
between WGCTA and CCTT to be .76, and between RJI and CCTT to be .46; after 
correcting for academic ability, the partial correlations for these pairs of instruments are 
.27, .54, and .27, respectively. Aware that facility with solving well-structured problems 
does not necessarily entail facility with solving ill-structured problems, Sternberg (1982) 
urges the development of better instruments to measure the latter and investigations into 
the relationship between these two skills. 

Overview of Research Findings 

Growth in Critical Thinking 

A generally consistent finding emerging from empirical studies on critical 
thinking is that significant growth on this cognitive dimension is both perceived (Astin, 
1993; Pace, 1974; Tsui, 1998) and experienced by college students (Dressel & Mayhew, 
1954; Gunn, 1993; Keeley, 1992; Keeley, Browne, Kreutzer, 1982; King, Wood & 
Mines, 1990; Klassen, 1983; Lehmann, 1963; McDonough, 1997; Mines, King, Hood, & 
Wood, 1990; Pascarella, 1989; Rykiel, 1995; Spaulding & Kleiner, 1992). Three of the 
studies in the sample, however, did not find a significant relationship between 
educational level and gains in overall critical thinking score (Anderson, 1988; Criner, 
1992; Drouin, 1992). In terms of degree of growth, a number of studies report freshmen 
year growth on tests for critical thinking to be in the vicinity of half a standard deviation. 
Such a finding emerges from the series of studies conducted by Dressel and Mayhew 
(1954), which some consider to be among the most comprehensive investigations of 
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critical thinking (McMillan, 1987; Pascarella & Terenzini, 1991). On a test of critical 
thinking in the social sciences administered to over 1,700 students at the beginning and 
end of freshmen year, statistically significant gains of about half a standard deviation (19 
percentile points) or better were obtained at each of the eleven participating institutions. 
On a test of science reasoning and understanding administered to 470 students, Dressel 
and Mayhew found statistically significant freshmen year gains of about half a standard 
deviation at six of the seven institutions tested. And, on a general measure for critical 
thinking (57 items designed to assess one’s ability to define a problem, select information 
pertinent to the problem, recognize assumptions, formulate relevant hypotheses, and draw 
valid conclusions) administered to 1,000 freshmen, statistically significant freshmen year 
gains of about half a standard deviation or larger were obtained at all seven participating 
institutions. Similarly, in a study by Pascarella (1989) in which a matched sample of 47 
students were employed, college freshmen year attendance was associated with a gain of 
.44 of a standard deviation on the WGCTA. In a separate study by Pascarella, Bohr, 
Nora, and Terenzini (1995), end of freshmen year gains of .41 and .24 of a standard 
deviation on the Collegiate Assessment of Academic Proficiency (C AAP) were attributed 
to full-time and part-time students, respectively. 

Generally speaking, research reveals more years of education is associated with 
higher scores on tests for critical thinking. Yet, there is research evidence to indicate that 
performance on the whole is poor. Norris (1985) reports that competence in critical 
thinking is reportedly lower than it should be at every level of schooling. According to 
Kuh (1995), many students who progress through the school system simply are not 
acquiring the ability to think critically. In a study by Keeley, Browne, and Kreutzer 
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(1982), in which critical thinking was measured though students’ abilities to analyze 
articles through an essay response format, seniors outperformed freshmen. Yet, the 
absolute level of performance displayed by seniors revealed “major deficiencies” in 
critical thinking skills. For instance, 40-60% of the senior group could not provide even 
a single example of a logical flaw, significant ambiguity, or misuse of data, when asked 
to evaluate a written passage containing several such errors. Using the same data source, 
Keeley (1992) analyzed freshmen and senior responses on a task to identify assumptions 
appearing in an essay and found “poor performance” displayed by both freshmen and 
seniors. Logan (1976), in a study involving 874 sociology students, concluded that “very 
low” scores were attained by students at each educational level studied. 

Research evidence also suggests that college students make the greatest gains in 
critical thinking during their freshmen year (Lehmann, 1963), which is consistent with 
research findings on general cognitive growth (Pascarella & Terenzini, 1991). Moreover, 
the present analysis reveals that freshmen are more likely than sophomores, juniors, and 
seniors to participate as subjects in studies on critical thinking. Hence, it is quite possible 
that generalizations made from study findings on critical thinking may at times lead to an 
overly optimistic interpretation of the overall impact of college on growth in critical 
thinking. From these considerations a provocative question arises: why do subsequent 
college years fail to rival freshmen year in its efficacy to enhance students’ critical 
thinking? Such an important question has been given little attention by researchers. 
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Instructional Factors 

A large proportion of the research studies conducted on critical thinking pertain to 
specific pedagogical methods. In the sample utilized for this review, approximately 29% 
of the studies investigate the effect of some pedagogical factor. Mixed results have 
emerged from these studies. In comparison to courses taught in a more traditional 
manner, greater gains in critical thinking scores were found for courses with an 
instructional paradigm emphasizing problem solving or critical thinking (Bailey, 1979), 
an instructional protocol emphasizing searching for meaning through reading, writing, 
and class participation (Gibson, 1985), an emphasis on inquiry and higher-order thinking 
(Suksringarm, 1976), out-of-class assignments designed to increase critical thinking 
(Eason, 1986), and attempts to elicit a high level of student participation, instructor 
encouragement, and peer-to-peer interaction (Smith, 1977). A number of instructional 
factors are positively related to self-assessed growth in critical thinking (Astin, 1993; 
Tsui, 1998), including having a paper critiqued by an instructor, conducting independent 
research, working on a group project, giving a class presentation, and taking essay 
exams; negatively related to this outcome is taking multiple-choice exams. 

Nonsignificant effects have been found, however, for courses stressing 
argumentation and discussion (Beckman, 1956), critical thinking objectives (Dressel & 
Mayhew, 1954), student-centered active learning (Forbes, 1997), inquiry and higher- 
order thinking (Hayden, 1978), reflective teaching (Phelps, 1987), as well as for courses 
employing such teaching techniques as the Guided Design (Hancock, 1981), self-paced 
instruction (Hardin, 1977), a values clarification approach (Jones, 1974), independent 
laboratory investigation (Norton, 1985), special manner of student grouping (Fishbein, 




10 



8 



1975), and alternative modes of instructions presentation (Coscarelli & Schwen, 1979). 
In a study in which political science students were exposed to either traditional lecture, an 
implicit model of critical thinking instruction, or an explicit model of critical thinking 
instruction. West (1994) found that in none of the sections did students improve their 
critical thinking skills to a statistically significant degree. Yet, statistically significant 
positive correlations were detected between WGCTA score and number of writing 
assignments completed (a component of the implicit model) and between WGCTA and 
number of critical thinking assignments completed (a component of the explicit model). 
Cases in which significant effects were not detected for instructional approach have 
caused some to doubt whether teaching method is relevant to the enhancement of critical 
thinking skills. Others maintain that failed attempts to establish differential effects on 
various modes of instruction are likely to be caused by limitations found with traditional 
research approaches (Smith, 1977). 

Curriculum Factors 

A significant portion of studies on critical thinking address effects of curriculum. 
In this sample approximately 26% of the studies include some factor relating to 
curriculum. Dressel and Mayhew (1954) found that the greatest student gains in critical 
thinking occurred at colleges with courses specifically devised for general education 
purposes. These colleges commonly involved the study of basic liberal arts and sciences 
in an integrative fashion, and emphasized basic integration and synthesis of knowledge 
rather than discrete bits of specialized information. It is believed that these institutions 
devote greater attention to developing general intellectual skills and orientations such as 
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analytical and critical skills, and tolerance for ambiguity (Gaff, 1983). Winter, 
McClelland, and Stewart (1981) likewise concluded that a curriculum that is focused on 
an integrative theme encompassing different disciplines leads to greater growth in critical 
thinking. The students who were exposed to the integrative program began with higher 
scores (1.66 versus 1.22) and achieved greater gains (an increase of .50 versus .08) than 
did the control group. These conclusions are congruent with findings by Astin (1993) in 
which students’ self-reported growth in critical thinking is positively related to taking 
interdisciplinary courses, and attending an institution with a humanities orientation and 
where there is heavy faculty involvement in teaching general education courses. In a 
study involving 431 college students, Beckett (1996) found that programs stressing 
experiential learning processes and self-directed learning processes positively influenced 
student scores on the WGCTA. Annis and Annis (1979), in a study involving 121 
students enrolled in Introduction to Philosophy, Logic, Ethics, and a nonphilosophy 
control class, found that when compared to the control group the Logic course had a 
consistent impact on certain aspects of critical thinking while the other philosophy 
courses did not. 

Courses or programs specifically designed to improve critical thinking have 
demonstrated mix results. Logan (1976) found superior skills were displayed by students 
who had enrolled in a sociology course with the explicit goal of teaching students to think 
critically, logically, and more scientifically. Moll and Allen (1982) discovered that 
students enrolled in a biology course specifically designed to improve both biology 
content knowledge and critical thinking skills made significant improvements on a 
locally developed instrument measuring both dimensions. However, no significant gains 
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were found for a special foundation research course on critical thinking (Gressler, 1976), 
or a psychology course designed specifically to enhance critical thinking (Lyle, 1958). 
A trail of mix findings regarding courses on critical thinking is not surprising when one 
considers how coursework with the same objectives and which seemingly address the 
same subject matter may differ vastly from one another across institutions in both 
approach and results. Halpren (1993) cautions that researching the effects of courses on 
critical thinking can be tricky. This is because a comparison of students who have and 
have not taken the course represents a methodological flaw in so far as such courses are 
usually required in the general education program at those colleges and universities 
where they are offered (i.e., those who have avoided taking the course differ in some 
fundamental way from those who take the course in the sense that they have somehow 
managed to avoid taking a required course). Some critics argue that critical thinking 
courses, mostly those based on informal reasoning, might be ineffectual to the extent that 
they encourage sophistry by teaching students skills that they can then use to rationalize 
their existing biases (Paul, 1982, 1986). According to Kurfiss (1988), there have been 
too few critical thinking programs implemented so far to reach a firm conclusion about 
their effectiveness. 

Disciplinary Field 

The investigation of effects of disciplinary major on critical thinking is found in 
approximately 16% of the studies included in this review. Generally speaking, a majority 
of studies have found no differences in critical thinking score across disciplines, and even 
in cases in which differences are detected they tend to dissipate after academic aptitude or 
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initial critical thinking ability is controlled for. Spaulding and Kleiner (1992) found that 
students with highly developed critical thinking skills are not self-selecting into particular 
college disciplines; majoring in liberal arts, social science, math/physical science, 
business and health science were not predictive of the student’s score on the CCTT. 
McDonough (1997), in a study of 240 community college students, found no disciplinary 
effects on WGCTA scores. Money (1997) did not detect any differences in CCTT scores 
by students pursuing business, music, or nursing studies. Moll and Allen (1982) also did 
not find any test score differences in critical thinking by biology and nonbiology students. 
Simon and Ward (1974) also did not find differences in WGCTA score between those 
enrolled in an arts program and those enrolled in a science program. Terenzini, Springer, 
Pascarella, and Nora (1995) found that the number of courses taken in different 
disciplinary areas was related to post-test scores in critical thinking at the end of 
freshmen year (number of math classes was negative, while number of science courses 
was positive), but these effects disappeared when precollege critical thinking was 
controlled. King, Wood, and Mines (1990) found that on the RJ1, social science majors 
earned higher scores than math science majors at the graduate level; no such difference 
was detected for the two groups at the undergraduate senior level. On both the WGCTA 
and CCTT, math science majors earned higher scores than social science majors, but such 
differences disappeared after controlling for academic aptitude. Disciplinary differences, 
however, were detected in a study by Gunn (1993) who found social science majors 
achieved higher scores than nonsocial science majors on the Wasatch Test of Critical 
Thinking. The number of courses taken in certain disciplinary areas such as history, 
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science, math, foreign languages, and ethnic studies has been found significantly and 
positively related to self-assessed growth in critical thinking (Astin, 1993; Tsui, 1998). 

Other Factors 

The relationship between critical thinking ability and a number of skills has been 
researched. In a study involving 302 students, Anderson (1988) found a significant 
relationship between moral judgment and critical thinking for the female group but not 
the male group. Brabeck (1980) concluded that critical thinking is necessary but not 
sufficient for the development of superior reflective judgment in an analysis of the test 
performance of 392 students which revealed that on the RJI those with high WGCTA 
scores outperformed those with low WGCTA scores. While those with low WGCTA 
scores scored uniformly low on the RJI, there was much greater variability in RJI scores 
by those with high WGCTA scores. Mines et. al (1990) showed that those who reasoned 
at higher stages of reflective judgment also demonstrated superior critical thinking skills. 
In a study involving 63 freshmen, Luckett (1991) found that critical thinking skills is 
positively related to both writing skills and reading skills. In analyzing scores on the 
CCTT, Steele Hanson (1986) found that more variation in critical thinking performance 
was accounted for by computer ability than mathematical ability. Research findings 
regarding critical thinking ability and college grades (or academic aptitude) are 
inconsistent as Hill (1995), Spaulding and Kleiner (1992), and Smith-Sanders (1997), 
uncovered evidence of a significant association while Criner (1982), Money (1997), and 
Pascarella (1989) did not. 
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Gains in critical thinking are significantly associated with attending college full- 
time (Pascarella, Bohr, Nora, & Terenzini, 1996) and residing on campus (Pascarella, 
Bohr, Nora, Zusman, Inman, & Desler, 1993). Economic status was found neither to be 
related to critical thinking skills (Criner, 1992) nor freshmen gains in critical thinking 
(Pascarella, 1989). And, while Criner (1992) found no correlation between parent 
education and critical thinking performance, Terenzini et. al (1995) found parent 
education level to be significantly associated with gains in critical thinking during 
freshmen year. 

Inconsistent findings also emerge concerning gender differences related to critical 
thinking. In several studies evidence of such differences was found (Gunn, 1993; 
Pascarella et. al, 1996; Simon & Ward, 1974), but a greater number have found no 
evidence of such differences (Anderson, 1988; Hill, 1995; Klassen, 1983; Moll & Allen, 
1982; Money, 1997; Pascarella, 1989; Terenzini et. al, 1995; Lehmann, 1963). Gender 
has been found to interact with number of philosophy courses taken (Annis & Annis, 
1979) and institutional type (Pascarella et. al, 1995) in predicting gains in critical 
thinking. With regard to critical thinking differences related to race, a modest but 
significant relationship was found in one study (Pascarella et. al, 1996) but not in three 
other studies (Hill, 1995; Pascarella et. al, 1989; Terenzini et. al, 1995). Race was found 
to interact with institutional type in the study by Pascarella et. al (1995) as greater gains 
in growth thinking was demonstrated by nonwhite students at 2-year colleges and by 
white students at 4-year colleges. 

Only more recently have out-of-class experiences received attention by those 
researching critical thinking. Employing the Collegiate Assessment of Academic 
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Proficiency (CAAP) and Pace’s College Student Experiences Questionnaire (CSEQ), 
Terenzini et al. (1995) found that, even with precollege critical thinking controlled, 
students’ out-of-class experiences contributed as much to gains in critical thinking at the 
end of the freshmen year as did their class-related experiences (2.9 percent and 2.5 
percent, respectively). The number of unassigned books read by students and the amount 
of time spent studying were found to positively affect critical thinking scores. 
Somewhat surprising, however, was the finding that gains in critical thinking scores were 
negatively related to students’ perceptions of the quality of their relationships with peers. 
Analyzing data from 24,847 college students, Astin (1993) found that being a guest in a 
professor’s home and attending an institution where there is a strong student orientation 
among the faculty are positively related to self-assessed growth in critical thinking. 

Utilizing a multiple institution sample of 1,054 students, Edison (1997) found that 
participation in out-of-class activities made a small but statistically significant 
contribution to gains on the CAAP. Faculty interaction and athletic or recreation 
facilities use were negatively associated with critical thinking scores. There is research 
evidence to support that hours spent outside of the classroom discussing racial or ethnic 
issues (Astin, 1993) or current issues with others (Smith-Sanders, 1997) positively 
influence students’ growth in critical thinking. Rykiel (1995) found that community 
college students who worked off campus 25 hours or more a week had higher critical 
thinking scores than those working less than 20 hours. Neither Criner (1992) nor 
Pascarella et. al (1993), however, found a relationship between number of hours worked 
and critical thinking performance. Jackson (1961) investigated the effects of 

participation in debate on critical thinking and found mixed results as debaters 
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experienced significantly greater gains than the control group at five colleges, but smaller 
gains for debaters than the control group at four other colleges. 

Evidence reviewed by Pascarella and Terenzini (1991) suggests that for such 
general cognitive outcomes as critical thinking the impact of any one academic or 
nonacademic experience may not be as important as students’ total level of engagement 
in the academic and social systems of the institution. This is congruent with previous 
research on the positive effect of general student involvement or quality of effort on 
cognitive development (Astin 1977, 1993; Pace, 1984). In a study by Pascarella (1989) 
various individual academic or social experience variables such as resident arrangement, 
time spent studying, extracurricular activities, and number of intellectually focused 
interactions with faculty and peers, were shown to have only trivial and statistically 
nonsignificant relations with critical thinking growth during the freshmen year. Yet, 
when these variables were combined into a composite variable representing social and 
intellectual involvement, a statistically significant positive association (partial r=.34) was 
found. 



Methodological Considerations 

Research Design 

Approximately 52% of the research studies in this sample employ both pre-test 
and post-test. That about 38% of the studies are not longitudinal in nature should be of 
some concern. Fifty-seven percent (13 out of 23) of the studies in this sample that 
attempt to measure student growth in critical thinking employ a cross sectional design. 
The use of such a methodological approach is problematic due to the effects of college 
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attrition. Researchers might be mistakenly inferring higher critical thinking test scores of 
upperclassmen as evidence that college attendance positively affects students’ abilities to 
think critically. Such a judgment is dubious because those who are adept at critical 
thinking are bound to be less likely to drop out of college than those who are not adept at 
critical thinking. Hence, in research studies utilizing a cross sectional design some of the 
“measured gains” in students’ critical thinking abilities might be inappropriately 
attributed to college exposure. Also in need of further attention in this area are the effects 
of maturation on critical thinking which occur independent of college attendance. This 
issue is not well researched since there appears to be only one study (Pascarella, 1989) 
wherein a college attending group is compared with a matched non-college-attending 
group, albeit using a relatively small sample of 47 participants. 

Even among those studies which are longitudinal a serious concern is raised by 
the typically short period that transpires between pre-test and post-test. In most cases 
treatment duration is usually no longer than a school year, and is frequently as brief as a 
quarter or semester term. Such a brief time span is problematic for it might not be long 
enough for true treatment effects to manifest. This methodological weakness may 
account for the generally inconsistent results that have emerged from research on the 
effectiveness of instructional techniques, and courses and programs specifically designed 
to enhance students’ critical thinking skills. Investigations of growth in critical thinking 
from enrollment in a specific type of course are further complicated by such potential 
mediating factors as instructor differences and simultaneous exposure to other 
coursework. 
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Another concern regarding research design is that most studies do not involve 
very large samples, nor multiple institutions. Approximately half of the studies employ 
samples of 200 or fewer students, while a fourth of the studies in this analysis employ 
samples of 100 or fewer students. Less than a third of the studies drew their sample from 
more than one institution. Without adequate research involving multiple institutions it is 
impossible to investigate the effects of institutional level variables. 

Measurement Instruments 

A somewhat limited range of measurement instruments has been used in critical 
thinking research. In terms of measuring students’ critical thinking abilities, the most 
commonly adopted approach involves standardized written multiple-choice tests. 
Benefits associated with the employment of standardized instruments have led the 
Watson-Glaser Critical Thinking Appraisal (used in 47% of the studies), Cornell Critical 
Thinking Test (used in 11% of the studies), and Collegiate Assessment of Academic 
Proficiency (used in 10% of the studies) to be the most frequently utilized in critical 
thinking research. Such measures, however, are not without weaknesses and limitations 
(see Berger, 1985; Helmstadter, 1985; Modjeski & Michael, 1983). A comparative 
evaluation of the WGCTA and CCTT were conducted by a panel of psychologist 
(Modjeski & Michael, 1983). Both tests obtained highly unfavorable valuations in the 
context of possible bias and lack of cross-validation efforts. Moreover, both measures 
were judged quite negatively in terms of stability of scores derived from the use of 
parallel forms across administrations. Norris (1985) cautions that critical thinking is 
extremely sensitive to context. McMillan (1987) warns that tests like the WGCTA are 
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too general to capture growth in certain areas of critical thinking, especially in the short- 
term, and may not be sufficiently domain specific to detect the influence of academic 
major or academic concentration. McPeck (1985) is critical of efforts that assume that a 
skill as complex as critical thinking can be adequately gauged by any multiple-choice 
exam. Berger (1985) alerts users not to overlook the fact that these purported thinking 
tests appraise critical thinking through reading (it is unclear as to whether a similar test 
administrated verbally would yield the same scores). 

Because multiple-choice tests have been criticized for measuring only product 
(answers to test questions) while neglecting process (how the application of critical 
thinking skills leads to answers), some researchers attempt to investigate process through 
analyzing verbalized or written responses to tasks designed to elicit critical thinking. For 
example, Winter and McClelland (1978) developed the Test of Thematic Analysis to 
elicit complex concept formation by asking students to read two groups of stories and 
then to write about their differences. Similarly, in studies by Keeley (1992) and Keeley 
et. al (1982) essay tests were administered wherein students were asked to read an essay 
then to evaluate the passages by answering some open-ended questions (e g., “identify 
and explain an ambiguity”). The focus here, according to Keeley et. al (1982), is “less on 
what the student can recognize among a number of different quality possibilities and 
more on what the student can find or generate for himself.” These approaches to 
studying critical thinking, however, are infrequent as only a handful of the studies in this 
sample employed such measures. 

Insight into critical thinking has been derived from students’ self-reported data. 
Pace (1974), Astin (1993), and Tsui (1998) have conducted research using students’ self- 
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assessed growth in critical thinking. Self-reported data has been much utilized in 
research pertaining to students’ general cognitive development (see Pascarella & 
Terenzini, 1991). Although some skepticism about the validity of a self-reported 
measure is expected, there is support for its usability. Bowen (1977) observed a general 
similarity between the results of cognitive outcome studies based on objective measures 
and those derived from students’ self-reports. And while self-reported cognitive abilities 
does not correspond perfectly with actual test scores, Pike (1996) found an adequately 
stable relationship existing between the two to warrant the use of self-reported data as 
general indicators of achievement. Moreover, both Astin (1993) and Anaya (1992) have 
found moderate positive correlations between self-reports and seemingly more objective 
measures of growth. In their review of the research literature on higher education, 
Pascarella and Terenzini (1991) concluded that student self-reports represent a 
“reasonable, if not totally adequate, indicator of cognitive growth.” Others advocate the 
use of indirect indicators such as self-reported data primarily because of the extreme 
difficulty of producing direct indicators (Ewell and Jones, 1993). 

It is quite clear that no single measure of critical thinking is perfect. In selecting 
the most appropriate instrument to measure the ability to think critically, one needs to 
seek the most suitable match between the specificity of the research question at hand and 
the strengths, weaknesses, and distinct characteristics of various instruments. Given that 
any choice made will entail limitations, advancement of knowledge becomes all the more 
reliant about the ongoing accumulation and constant challenge of evidence drawn from a 
diversified bounty of studies. Since a majority of studies on critical thinking focus on 
classroom factors, future research efforts should consider the use of such qualitative 
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methods as classroom observations and participant interviews-investigative tools that 
have thus far been largely bypassed. 

Variables 

Clearly the primary focus of research on critical thinking has been on classroom 
experiences. And while such investigations are worthwhile, there needs to be more 
research involving out-of-classroom factors. Inquiry into the effects of any form of out- 
of-class experience on critical thinking skills development appears in only 9 out of the 62 
studies in the present sample. This is disconcerting since both personal experience and 
research suggest that a significant amount of student growth can be attributed to students’ 
out-of-class experiences. The potential effects of peer group influence and engagement 
in college activities outside the classroom have been little studied. Future research 
should also consider institutional level variables such as institutional type, faculty-student 
ratio, per-student instruction expenditure, and elements within the campus culture which 
may significantly impact student growth in critical thinking. 

Conclusion 

Although the virtue and imperative of critical thinking is widely espoused, there is 
surprisingly little known through empirical research about the development of critical 
thinking in college (Keeley & Browne, 1986). While we are fairly confident that 
students’ abilities to think critically grow during college, there is much less known about 
what actually enhances critical thinking. Given that studies on this subject tend to be 
somewhat homogenous to the extent that they pose common research questions, (certain 
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instructional techniques or curriculum offerings designed to enhance critical thinking are 
effective) and employ similar research tools (primarily standardized tests of critical 
thinking), there is a surprising amount of heterogeneity (inconsistencies) emerging from 
study results. This review highlights a number of concerns that arise concerning research 
conducted on critical thinking. Because relatively little substantial knowledge has 
emanated from past research on critical thinking, more exploratory research involving the 
simultaneous control of a wide spectrum of variables would be beneficial. Since our best 
understanding of critical thinking will not come from any single research endeavor, but 
rather the critical examination of findings from a range of research studies, diversifying 
research design, measurement instruments, and variables investigated will be integral to 
expanding our understanding about this complex skill that is likely to be affected by a 
mosaic of sources within the higher education setting. 
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